Meta-Rep 29 - 31 October 2024 | Munich
We did not find a significant correlation between variable X and variable Y (\(n = 18\) , \(r = .40\), \(p = .100\))
We found a significant correlation between variable X and variable Y (\(n = 1000\), \(r = .08\), \(p = .011\))
It is the smallest effect size associated with a chosen \(\alpha\) (therefore significant), given sample size, test chosen and direction of the hypothesis.
It would be ideal to plan \(n\) to reach a power of 80%. Often we are in a scenario where it was not possible to pre-plan sample size, either for limited resources or because we accessed a large database.
In the first case optimal power cannot be reached
In the second case even small effects reach statistical significance
In the first example the correlation was not significant, but the critical effect size was \(r = ±.468\). Are we sure that a correlation of .30 is not relevant?
In the second example the critical effect size was \(r = ± 0.062\), depending on the construct under investigation such a small effect might not have practical meaning.
Before conducting the study:
In front of the dead corpse:
And more..
Our package helps to compute critical effect sizes for correlations, group comparisons, linear regressions and meta-analysis.
Reality is always more complex than a correlation of a linear model
We will work on the implementation of critical effect sizes for more complex models (i.e. Structural Equation Modeling)
Of course it would be better to pre-plan sample size by formalizing a plausible effect size BEFORE conducting the study
But.. when it is not possible to plan \(a\) \(priori\) the study, critical effect sizes allow to undertand beforehand the limitations of the study design, without having to specify a plausible effect.
Our motto is TBT:
Thinking Before Testing!
Here you can find our package, examples on how to use the functions and the draft of the paper:
Ambra Perugini
ambra.perugini\(@\)phd.unipd.it
https://psicostat.dpss.psy.unipd.it/people.html